Intuitive Human-Robot Interaction Through Active 3D Gaze Tracking

One of the biggest obstacles facing humans and robots is the lack of means for natural and meaningful interaction. Robots find it difficult to understand human intentions since our way of communication is different from the way machines exchange their information. Our aim is to remove this barrier by creating systems that react and respond to natural human actions. In this research, we present a robotic system that identifies and picks up an arbitrary object in 3D space that a person is looking at. It is done through an active vision system that is able to understand the focus of attention of a user. Whenever the person is close enough, the gaze direction is determined and used to find the object of interest which is of unknown size, shape and color. A robot arm responds by picking up this object and handing it over to the person.